Skip to content

fix(python): accurate LLM hint and exit 124 on Monty time limit - #2833

Merged
chaliy merged 2 commits into
mainfrom
claude/project-thread-pxlkst-python-hint
Nov 5, 2026
Merged

chaliy merged 2 commits into
mainfrom
claude/project-thread-pxlkst-python-hint

Conversation

@chaliy

@chaliy chaliy commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Михайло · project thread

What changed

  • The embedded python3 LLM hint describes what Monty actually runs: the real stdlib list (json, datetime, unicodedata were missing), re disabled, simple classes only (no inheritance), no match, output via print(). A new test imports every module the hint names and checks the class/match/re claims, so the next Monty upgrade that changes them fails the build instead of silently misleading agents.
  • A Monty time-limit hit exits 124 with python3: execution timed out after Ns, keeping earlier output, the same as the CPython runtime and coreutils timeout. A script's own TimeoutError is still an ordinary exit-1 exception.

Why

Found reviewing a downstream integration (harnx): its docs had to warn users that bashkit's "no classes" hint is wrong, and its backend greps stderr for TimeoutError: time limit exceeded: to tell a timeout from a script error.

Before / After

Before:

$ bashkit -c "python3 -c 'print(1)
while True: pass'; echo rc=\$?"     # max_duration 200ms
1
Traceback (most recent call last): ...
TimeoutError: time limit exceeded: 200.1ms > 200ms
rc=1

After: stdout 1\nrc=124, stderr python3: execution timed out after 0.2s (timeout_limit_exits_124). Hint before: ... No HTTP/network. No classes. No third-party imports.; after: see Python::llm_hint, enforced by test_llm_hint_matches_runtime.

Risk

  • Low
  • Scripts checking for exit 1 on a Monty timeout now see 124.

Checklist

  • Tests added or updated
  • Backward compatibility considered

Generated by Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

@chaliy chaliy self-assigned this Nov 5, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 8a6974c Commit Preview URL

Branch Preview URL
Nov 05 2026, 04:38 AM

@bashkit-pr-reviewer bashkit-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Monty time-limit reporting change (exit 124 + preserved stdout) and the hint/knowledge accuracy updates. One correctness issue worth fixing; details inline.

Comment thread crates/bashkit/src/builtins/python.rs Resolved
@bashkit-pr-reviewer

bashkit-pr-reviewer Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Review summary (head 8a6974c)

Verdict: Re-reviewed after the rebase to 876421f. Same 5 files and stats (+155/-17); the reviewed code (flag-based timeout discriminator, hint, tests, docs, knowledge) is unchanged in substance from e166090. No new findings.

Findings: P0: none. P1: none new — python-timeout-prefix-spoof from the earlier head remains fixed (per-run time_limit_hit flag set only on ResourceError::Time, plus the spoof regression test). P2: none.

Earlier notes (unchanged): head hint documents "no match statements", consistent with test + knowledge; the "matches the CPython runtime" wording remains unverifiable from the checkout but harmless.

The hint said "No classes" after Monty 0.0.19 added simple classes and
never mentioned that `re` is disabled. Rewrite it and pin it with a test
that imports every module it names and checks the class/match claims.

A Monty time-limit hit returned exit 1 and a TimeoutError traceback, so
callers had to parse stderr to tell a limit from a script error. Report
it like the CPython runtime: exit 124, 'python3: execution timed out'.
A script could raise TimeoutError('time limit exceeded: ...') and be
reported as a limit hit (exit 124). Record in the tracker when its own
time check fires and key the exit-124 path on that.
@chaliy
chaliy force-pushed the claude/project-thread-pxlkst-python-hint branch from e166090 to 8a6974c Compare November 5, 2026 04:37
@chaliy
chaliy merged commit fe82ced into main Nov 5, 2026
57 checks passed
@chaliy
chaliy deleted the claude/project-thread-pxlkst-python-hint branch November 5, 2026 04:59
@chaliy chaliy mentioned this pull request Nov 10, 2026
2 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant